Universal Access To All Knowledge
Home Donate | Store | Blog | FAQ | Jobs | Volunteer Positions | Contact | Bios | Forums | Projects | Terms, Privacy, & Copyright
Search: Advanced Search
Anonymous User (login or join us)
Upload

Reply to this post | See parent post | Go Back
View Post [edit]

Poster: Michael Birk Date: Mar 8, 2005 6:22am
Forum: etree Subject: Re: Why doesn't the archive allow files to resume?

Well, my apologies if I am re-hashing an old conversation. However, a few points:

It should be possible to do scalable, on-the-fly zipping that supports resumption. There is no need to use ZIP file compression, since these audio files are already compressed (with MP3, OGG, Shorten, or Flac). Without the ZIP compression, it should not be CPU-intensive.

HTTP resumption pretty much works for all clients, assuming the server supports it. As we are discussing, it is a bit tricky to implement for dynamic content, but certainly not impossible.

There are some advantages to HTTP over FTP for content distribution (even large files). In particular, caching is much more straightforward, since the HTTP protocol specifically supports it.

I sent an email last night to info@archive.org offering to help with the on-the-fly-zip. Any chance you will take me up on the offer? If I just implement it as, say, a PHP script, could you use it?

thanks,
mcb

p.s. The petabox looks pretty cool! :-) However, if you store the .zip files, is it really a 500-gigabox?

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or StaffBrad Leblanc Date: Mar 8, 2005 10:50am
Forum: etree Subject: Re: Why doesn't the archive allow files to resume?

It should be possible to do scalable, on-the-fly zipping that supports resumption. There is no need to use ZIP file compression, since these audio files are already compressed (with MP3, OGG, Shorten, or Flac). Without the ZIP compression, it should not be CPU-intensive

Well, I guess I'm still at the point where I don't see the benefit. If space isn't an issue, what does the on-the-fly stuff gain us?

I sent an email last night to info@archive.org offering to help with the on-the-fly-zip. Any chance you will take me up on the offer? If I just implement it as, say, a PHP script, could you use it?

I responded to that around 2 or 3 this afternoon Michael. Not sure why you haven't seen it yet. Let me know if I need to resend.

If I just implement it as, say, a PHP script, could you use it?

I'm not the person that will be implementing it (I'm just a librarian and middleman for the real engineers), but I guess if you can convince me of why we should use on-the-fly then I will send it to them. If we're retiring it to free up resources (CPU), what does keeping it around help with?

We appreciate your offer to help.

The petabox looks pretty cool! :-) However, if you store the .zip files, is it really a 500-gigabox?

No, it's a 500,000 gigabox, or a 500 terabox. :) And when that fills up in 10-15 years we talk about rolling in another bigger one. We'll see...

-Brad

This post was modified by Brad Leblanc on 2005-03-08 18:50:31

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or StaffDiana Hamilton Date: Mar 8, 2005 11:07pm
Forum: etree Subject: Re: Why doesn't the archive allow files to resume?

No, it's a 500,000 gigabox, or a 500 terabox. :) And when that fills up in 10-15 years we talk about rolling in another bigger one. We'll see...

Gosh, remember when we had etree01 and etree02 and imagining in this forum when we'd be up to etree38 and etree39... "yeah, that will be really cool". Same feeling here and now. :)